About the User: David had some familiarity with Archive.org and had used both v1 and v2 on 
multiple occasions to perform research. 


Key findings: 

• The user was unable to complete the task of filtering a collection by language. 

• The user had a lot of difficulty understanding collection hierarchy and had a hard time 
differentiating between search results, the homepage and collection pages. 

• User expected type ahead or search suggestions on search bars on the website. 

• The user was unable to complete the task of sorting the collection by date. 

• User expected clicking on the collection header to remove applied filters. 

• Book reader vs. PDF view vs. Item Page was confusing for the user. 

• User avoided use of the navigation bar and primarily used the homepage for 
navigation. 

• User was unsure from tile what types of media a collection might hold. 


Testing Notes 

Task 1: Please go to Archive.org 

Task 2: Please go to the ‘American Libraries’ Collection 

Task rating: 5 (not easy) 

Step A: Complete the task above 

David began by scrolling down the home page to look for the American Libraries collection. He 
stated that he normally does not scroll very far down the page and was surprised to see the long 
list of collections. 

Next he tried using the central search box on the homepage and searched for American 
Libraries 

Step B: What are vour first impressions of this page? 

David was not sure if the collections listed on the American Libraries collection were part of the 
collection or not. He seemed to think that potentially he was in a search results page, but he did 
not feel confident that he was in a collection. 

Task 3: How do you think the items in this collection are currently ordered? 

Task rating: User could not complete this task 

Step A: Complete task above 

David was able to locate the sort bar quickly but he had trouble seeing which option was 
currently selected. He also thought that perhaps the filters and the sort bar affected different 
parts of the search results area. While spending time on the American Libraries collection he 
continued to be confused about which part of the interface was a part of the collection and 



which part was simply a display of collections / search results. David even went back to the 
homepage to see if the collection tiles were represented in the same way on the American 
Libraries collection as on the homepage. 

Step B: How would you sort items in this collection bv the date they were added to Archive.org. 
with most recently uploaded items first? 

David was unable to complete this task. 


Task 4: How would you view only items that are in English in this Collection? 

Task rating: User could not complete this task 

David began by looking in the Topics area for something that indicated English. He could not 
find the Language options so he used the search box within the collection along with the media 
filter of texts. The search term he used was ‘English Language,’ which yielded results in a 
number of languages, not just English. 

Organize the books bv date published 

David was able to apply the Date Published sort, but he did not feel like he had appropriate 
feedback from the system to be sure that he had actually applied the sort. David was not sure 
how to modify the sort by ascending and descending order (both in list view and in tile view). 

Task 5: How would you find an image of “Children Reading” in this collection? 

Task rating: 

Task 6: Item Details Page 

No task rating 

David opened a book item and expected to see the cover (what was featured on the collection 
title) in the media theater. David had no trouble flipping through the pages in the media theater 
but needed a larger version of the book to read the text, so he clicked into the book reader. 
Once in the book reader he still had trouble reading the page so he zoomed in and was 
frustrated that the commands for flipping pages had changed from a click to a drag. David also 
had some trouble locating the scrollbar, and wished that it was possible to search for a specific 
page number. 

When asked to download the book he was reading in the book reader David began by looking 
for a download button, he did find the informational popup but after clicking on the PDF link he 
was able to save the PDF to the desktop. 


From the PDF view David was not able to navigate back to the book reader or the details page. 
Once guided he used the back button and got back to the book reader view. Once in the book 
reader view he clicked on the lA logo and was sent back to the homepage. With further 





guidance he went back to the book reader, clicked on the text link and finally returned to the 
item page. 


Task 7: Where would you go to find out more information about one of the collections? 

Task rating: 

David began by scrolling down the item page and quickly found the collection field. However, he 
was still unsure of the provenance of the item or the collection’s relationship to the Internet 
Archive. 

Task 8: Please find and go to one of the Archive’s software collections 

Task rating: 

David was familiar with some of the Archive’s software collections, so he began his search by 
returning to the homepage and browsing for an appropriate collection. David had some 
questions of what would fall under software, for instance OS emulation and video games were 
genres he wasn’t sure would fit under software. He recognized a Mario avatar on the home 
page and clicked into the ‘Speed Runs’ Collection, which is actually a Movie Collection. 

Once in the collection he still did not notice that the collection did not contain software. 

Task 9: Where would you go to find out more information about this collection? 

Task rating: 8 (easy) 

Step A: Where would you find more information on the collection 

David’s first inclination was to click on ‘show details,’ however he quickly found the About tab. 

Bonus 

David next tried to find what he was initially looking for, emulation of Atari games. David began 
this task by using the search term ‘Atari Game’ however he felt very frustrated because he 
wanted to find a collection, not individual items. He returned to the homepage and began to 
scroll through the list of Top Collections and became very frustrated. 

He then noticed the icons on the homepage above the search bar, he clicked the icon (which he 
noted felt very small) and found the software collection. 

Follow-up Questions 

1. How did this interface match with vour expectations of using Archive.org. what parts 
were easy to use and behaved as expected? 

David felt just as frustrated with Archive.org V2 as he did with V1 . Cverall he persists 
with using Archive.org because of the content. Generally he uses search engines 
(Google, Duck Duck Go) to locate items in the Archive. 




2. In what wavs did this interface not match vour expectations of using Archive.org. what 
parts were hard to use, or weren’t quite clear to you? 

David was not sure how the Archive is organized, and why collection provenance is so 
exposed. He would prefer to have more stuff hidden under the hood and to use search 
to surface what he is looking for. 

3. If you could change one thing about Archive.org. what would you change? 

David would have the Wayback Machine’s search bar less prominent than it currently is. 
He has had trouble before accidentally using this search box in the past. 

4. Have you ever created an account on Archive.org. or would you? Why or why not? 

David made an account but he doesn’t generally use his account. 


